High Efficiency Video Coding (HEVC) is a new coding standard that has been developed in recent years. In the High Efficiency Video Coding (HEVC) system, the fixed-size macroblock of H.264/AVC is replaced by a flexible block, named coding unit (CU). Pixels in the CU share the same coding parameters to improve coding efficiency. A CU may begin with a largest CU (LCU), which is also referred as coded tree unit (CTU) in HEVC. In addition to the concept of coding unit, the concept of prediction unit (PU) is also introduced in HEVC. Once the splitting of CU hierarchical tree is done, each leaf CU is further split into one or more prediction units (PUs) according to prediction type and PU partition. Several coding tools for screen content coding have been developed. These tools related to the present invention are briefly reviewed as follow.
Palette Coding
During the development of HEVC screen content coding (SCC), several proposals have been disclosed to address palette-based coding. For example, a palette prediction and sharing technique is disclosed in JCTVC-N0247 (Guo et al., “RCE3: Results of Test 3.1 on Palette Mode for Screen Content Coding”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Vienna, AT, 25 Jul.-2 Aug. 2013 Document: JCTVC-N0247) and JCTVC-O0218 (Guo et al., “Evaluation of Palette Mode Coding on HM-12.0+RExt-4.1”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Geneva, CH, 23 Oct.-1 Nov. 2013, Document: JCTVC-O0218). In JCTVC-N0247 and JCTVC-O0218, the palette of each color component is constructed and transmitted. The palette can be predicted (or shared) from its left neighboring CU to reduce the bitrate. All pixels within the given block are then coded using their palette indices. An example of encoding process according to JCTVC-N0247 is shown as follows.    1. Transmission of the palette: the color index table (also called palette table) size is first transmitted followed by the palette elements (i.e., color values).    2. Transmission of pixel values: the pixels in the CU are encoded in a raster scan order. For each group of one or more pixels, a flag for a run-based mode is first transmitted to indicate whether the “copy-index mode” or “copy-above mode” is being used.            2.1 “Copy-index mode”: In this run-based mode, a palette index is first signaled followed by a value “palette_run” (e.g., M) representing the run value. The “copy-index mode” is also referred as “index” mode in some literature. The term “palette_run” may also be referred as “pixel_run” or simply “run” in this disclosure. The run value indicates that a total of M consecutive samples are all coded using run mode. No further information needs to be transmitted for the current position and the following M positions since they have the same palette index as that signaled in the bitstream. The palette index (e.g., i) may also be shared by all three color components, which means that the reconstructed pixel values are (Y, U, V)=(paletteY[i], paletteU[i], paletteV[i]) for the case of YUV color space.        2.2 “Copy-above mode”: In the copy-above mode, a value “palette_run” (e.g. N) is transmitted to indicate that for the current pixel and the following N pixels, the palette index is the same as the corresponding palette index in the row above.            3. Transmission of residue: the palette indices transmitted in Stage 2 are converted back to pixel values and used as the prediction. Residue information is transmitted using HEVC residual coding and is added to the prediction for the reconstruction.
Both the copy-index mode and copy-above mode are referred as “copy modes” in this disclosure. In the copy-index mode, the palette index of the leading pixel of the copy-index coded pixels is transmitted. The number of the copy-index coded pixels is also transmitted. At the decoder side, after determining the coding mode (i.e., copy-index mode) for the underlying pixels, only the information of the palette index of the leading pixel and the number of the copy-index coded pixels are needed to reconstruct the copy-index coded pixels. The pixels following the leading pixel can be reconstructed by “copying” from the predictor (i.e., the pixel on the left). The pixel value can be reconstructed using the palette table. In the copy-above mode, no palette index related to the predictors (i.e., the above pixels) needs to be transmitted since the above pixels have been reconstructed. The current pixels can be “copied” from the above pixels.
In JCTVC-N0247, palette of each component are constructed and transmitted. The palette can be predicted (shared) from its left neighboring CU to reduce the bitrate. In JCTVC-O0218, each element in the palette is a triplet, which represents a specific combination of the three color components. Furthermore, the predictive coding of palette across CU is removed.
Another palette coding technique similar to JCTVC-O0218 has also been disclosed. Instead of predicting the entire palette table from the left CU, individual palette color entry in a palette is predicted from the exact corresponding palette color entry in the above CU or left CU.
For transmission of pixel palette index values, a predictive coding method is applied on the indices as disclosed in JCTVC-O0182 (Guo et al., “AHG8: Major-color-based screen content coding”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Geneva, CH, 23 Oct.-1 Nov. 2013, Document: JCTVC-O0182). Three types of line modes, i.e., horizontal mode, vertical mode and normal mode are used for coding each index line. In the horizontal mode, all the indices in the same line have the same value. If the value is the same as the first pixel of the above pixel line, only line mode signaling bits are transmitted. Otherwise, the index value is also transmitted. In the vertical mode, it indicates that the current index line is the same with the above index line. Therefore, only line mode signaling bits are transmitted. In normal mode, indices in a line are predicted individually. For each index position, the left or above neighbors is used as a predictor, and the prediction symbol is transmitted to the decoder.
Furthermore, pixels are classified into major color pixels (with palette indices pointing to the palette colors) and Escape pixel according to JCTVC-O0182. For major color pixels, the pixel value is reconstructed according to the major color index (i.e., palette index) and palette table in the decoder side. For Escape pixel, the pixel value is further signaled in the bitstream.
Palette Table Signaling
In the reference software of screen content coding (SCC) standard, SCM-2.0 (Joshi et al., Screen content coding test model 2 (SCM 2), Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting: Sapporo, JP, July 2014, Document No.: JCTVC-R1014), an improved palette scheme is integrated in JCTVC-R0348 (Onno, et al., Suggested combined software and text for run-based palette mode, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting: Sapporo, JP, July 2014, Document No.: JCTVC-R0348). The palette table of previous palette-coded CU is used as a predictor for current palette table coding. In palette table coding, the current palette table is signaled by choosing which palette colors in the previous coded palette table (palette predictor) are reused, or by transmitting new palette colors. The size of the current palette is set as the size of the predicted palette (i.e., numPredPreviousPalette) plus the size of the transmitted palette (i.e., num_signaled_palette_entries). The predicted palette is a palette derived from the previously reconstructed palette coded CUs. When coding the current CU as a palette mode, those palette colors that are not predicted using the predicted palette are directly transmitted in the bitstream (i.e., signaled entries).
An example of palette updating is shown as follows. In this example, the current CU is coded as palette mode with a palette size equal to six. Three of the six major colors are predicted from the palette predictor (numPredPreviousPalette=3) and three are directly transmitted through the bitstream. The transmitted three colors can be signaled using the exemplary syntax shown below.
num_signaled_palette_entries=3
for(cIdx=0; cIdx<3; cIdx++)//signal colors for different components
for(i=0; i<num_signaled_palette_entries; i++)
palette_entries[cIdx][numPredPreviousPalette+i]
Since the palette size is six in this example, the palette indices from 0 to 5 are used to indicate the major color entries in the palette color table. The 3 predicted palette colors are represented with indices 0 to 2. Accordingly, three new palette entries are transmitted for indexes 3 through 5.
In SCM-2.0, if the wavefront parallel processing (WPP) is not applied, the palette predictor table is initialized (reset) at the beginning of each slice or at the beginning of each tile. If the WPP is applied, the last coded palette table is not only initialized (reset) at the beginning of each slice or at the beginning of each tile, but also initialized (reset) at the beginning of each CTU row.
Palette Prediction Coding
In JCTVC-O0218 and JCTVC-N0247, the palette prediction flags are used for predicting the palette from the last coded palette CU. One bit is used for each entry. If the palette prediction flag of a palette entry is 1, this palette is reused for the current CU.
In JCTVC-R0348, a different binarization method for the codeword is disclosed, where the zero-runs between each predicted palettes are coded. The codeword N means there are N zeros before the next palette reuse flag that is 1. For example, a “0” means that the next palette is reused (i.e., no 0 between the current and next reuse flag 1). A “1” means the second next palette is reused. Another example for the zero-run based palette prediction is shown in FIG. 1 for a palette prediction table with 10 palettes. The palette at palette table entries corresponding to 0, 2, 3, 5, and 8 are reused by the current CU. The zero-runs of the predicted palettes are {0, 1, 0, 1, 2} respectively.
The binarization method can be Exponential-Golomb code with order-K (EG-K code), truncated Exponential-Golomb code with order-K (truncated EG-K code), or N-bit Truncated Unary code+EG-K code. The K can be 0, 1, 2, or 3. The N can be 0, 1, 2, or 3.
In order to signal the last used palette, a method to insert an end-code (EC) into the codeword is disclosed in JCTVC-R0228 (Seregin, et al., Non-SCCE3: Run-length coding for palette predictor, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting: Sapporo, JP, July 2014, Document No.: JCTVC-R0228). The zero-runs that are equal to or larger than EC will be increased by one. In the decoder side, if the decoded zero-run is equal to EC, it means that there is no other predicted palette. If the decoded zero-run is larger than EC, the actual zero-run is the decoded zero-run minus one. For example, the EC can be set to 1 and any zero-run larger than or equal to 1 will be increased by 1. Accordingly, the encoded/decoded codewords for the example in FIG. 1 becomes {0, 2, 0, 2, 3, 1} as shown in FIG. 2.
In the existing palette coding, the decoder may encounter problems when the predictor is an Escape pixel. For example, if the copy-above mode is selected and an above pixel is an Escape pixel, a current pixel below the Escape pixel may not be reconstructed properly. According to the existing copy-above mode, only the run value is signaled. An Escape index can be assigned to the current pixel below the above Escape pixel. However, since the Escape value is not transmitted, the current pixel below the above Escape pixel cannot be reconstructed. Accordingly, it is desirable to develop methods to overcome the issue associated with a predictor being an Escape pixel.